Shared input and recurrency in neural networks for metabolically efficient information transmission

Shared input to a population of neurons induces noise correlations, which can decrease the information carried by a population activity. Inhibitory feedback in recurrent neural networks can reduce the noise correlations and thus increase the information carried by the population activity. However, the activity of inhibitory neurons is costly. This inhibitory feedback decreases the gain of the population. Thus, depolarization of its neurons requires stronger excitatory synaptic input, which is associated with higher ATP consumption. Given that the goal of neural populations is to transmit as much information as possible at minimal metabolic costs, it is unclear whether the increased information transmission reliability provided by inhibitory feedback compensates for the additional costs. We analyze this problem in a network of leaky integrate-and-fire neurons receiving correlated input. By maximizing mutual information with metabolic cost constraints, we show that there is an optimal strength of recurrent connections in the network, which maximizes the value of mutual information-per-cost. For higher values of input correlation, the mutual information-per-cost is higher for recurrent networks with inhibitory feedback compared to feedforward networks without any inhibitory neurons. Our results, therefore, show that the optimal synaptic strength of a recurrent network can be inferred from metabolically efficient coding arguments and that decorrelation of the input by inhibitory feedback compensates for the associated increased metabolic costs.


Introduction
The efficient coding hypothesis poses that neurons evolved due to evolutionary pressure to transmit information as efficiently as possible [1].Moreover, the brain has only a limited energy budget, and neural activity is costly [2,3].The metabolic expense associated with neural activity should, therefore, be considered, and neural systems likely work in an informationmetabolically efficient manner, balancing the trade-off between transmitted information and the cost of the neural activity [4,5,6,7,8].
The principles of information-metabolically efficient coding have been successfully applied to study the importance of the excitation-inhibition balance in neural systems.It has been shown that the mutual information between input and output per unit of cost for a single neuron is higher if the excitatory and inhibitory synaptic currents to the neuron are approximately equal if the source of noise lies in the stochastic nature of the voltage-gated Na + and K + channels [9].In a rate coding scheme, where the source of noise lies in the random arrival of presynaptic action potentials, the mutual information per unit of cost has been shown to be rather unaffected by the increase of pre-synaptic inhibition associated with an excitatory input [10].
However, the balance of excitation and inhibition is likely to be more important in the context of recurrent neural networks than in the context of single neurons.In recurrent neural networks, the inhibitory input to neurons associated with a stimulus [11] arises as inhibitory feedback from a population of inhibitory neurons.The inhibitory feedback prevents a selfinduced synchronization of the neural activity [12] and reduces noise correlations (correlations between neurons calculated across trials of the same stimulus) induced by shared input to neurons in the population [13,14,15].If noise correlations have the same sign as signal correlations (correlations between neurons calculated across different stimuli), then noise correlations are detrimental to information transmission by neural populations [16,17,18].Information is likely transmitted by the activity of a population of neurons instead of a single neuron [19], therefore, when studying the effect of excitation-inhibition balance on information transmission, it is essential to consider the context of neural populations.In the case of a population of neurons tuned to the same stimulus, positive noise correlations decrease the information content in the population.
Several studies have analyzed the effect of noise correlations on information transmission properties [16,17,20].However, these studies did not analyze the relationship between the noise correlations and the metabolic cost of neural activity.In our work, we consider a computational model of a small part of the sensory cortex and the noise correlations caused by shared connections from an external thalamic population.The noise correlations may then be reduced by inhibitory feedback, which, however, increases the cost of the neural activity [10].
Our point of interest is the trade-off between improved information transmission due to lower noise correlations and the increase in metabolic costs due to stronger inhibitory feedback.

Constrained information maximization in a simple linear model
In order to gain an insight into what affects the information-metabolic efficiency of a neural population, we first solve the problem for a simple linear system.The mean response of the system is given by γ(λ ext ) = gλ ext , where λ ext is the stimulus and g is the gain of the system.We measure the trial-to-trial variability of the response with the Fano factor, defined as where N is a random variable representing the response n of the network to some stimulus.In this section, we assume the Fano factor to be constant, and we assume that the output is continuous and normally distributed.Therefore, the input-output relationship is described by the conditional probability We assume that the cost of the activity w(λ ext ) depends linearly on the input: where W 0 is the cost of the resting state.We treat the input λ ext as a random variable Λ with probability distribution function p (λ ext ).We can then calculate the average metabolic cost as The mutual information between the input and the output I(Λ; N) is calculated as where f(n|λ ext ) is the probability distribution function of N given that Λ = λ ext , p(λ ext ) is the input probability distribution, i(λ ext ; n) is the amount of information that an observation of n spikes gives us about the stimulus λ ext , i(λ ext ; N) is then the average amount of information we get from the input λ ext , q p (n) is the marginal output probability distribution.The capacity-cost function C(W) is the lowest upper bound on the amount of mutual information (in bits) achievable given the constraint that W p < W: The information-metabolic efficiency E is then the maximal amount of mutual information per molecule of ATP between the input and the output: The capacity-cost function can be obtained numerically with the Blahut-Arimoto algorithm [21].The information-metabolic efficiency can be conveniently obtained directly with the Jimbo-Kunisawa algorithm [22,23].However, if the Fano factor is very small, a lower bound on the capacity-cost function can be found analytically [24,25].In the low noise approximation, the optimal input distribution maximizing the mutual information constrained by metabolic expenses W is given by where J(λ ext ) is the Fisher information and λ 1 and λ W are the Lagrange multipliers which can be obtained from the normalization condition: and the average metabolic cost constraint (Eq 4).In the second-moment approximation [26,27], the Fisher information is given by where μ(λ ext ) is the mean response to the external input λ ext , μ 0 (λ ext ) is the derivative, and σ exc (λ ext ) is the standard deviation of the spike counts at input intensity λ ext .The low noise estimate on the capacity-cost function is then the information-metabolic efficiency can be conveniently obtained directly with the Jimbo-Kunisawa algorithm [22,23].
In the case of our simple linear system the Fisher information (Eq 14) is and the probability distribution derived from the low-noise approximation (Eq 12) is then ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffiffi 1 2pe After applying the normalization conditions (Eqs 4 and 13) and using Eq (15) we obtain the lower bound on the capacity-cost function: where w AP is the cost of increasing the output intensity by one action potential.The gain g, cost scaling w 0 , and Fano factor FF cannot be considered constant for real neural populations.However, Eq (18) provides an insight into the importance of these properties, which we will study numerically for a more realistic neural system.
In the following, we use Next, we analyze the information-metabolic efficiency of a recurrent spiking neural network, consisting of 800 excitatory and 200 inhibitory neurons.This network may represent a small area in the cortex, tuned to the same external stimulus, such as approximately a sphere of a 145 μm radius in the rat barrel cortex, which comprises only a small fraction of a single barrel [28,29].In such case, the external input is the input from a single barreloid in the thalamus.We assume that the role of this subnetwork is to process information about the stimulus intensity.We analyze the information-metabolic efficiency in two extreme cases of the readout of the network.First, we assume that the output of the network is read out as the summed rate of all the neurons in the network, and second, we assume that the brain acts as an efficient unbiased decoder with access to the rate of each neuron.In each case, we calculate the rate of each neuron as the number of fired spikes in a time window ΔT = 1 s.

Inhibitory feedback decorrelates the neural activity
In our model, 1000 external neurons randomly connect to the excitatory and inhibitory subpopulations with a connection probability P ext (Fig 1).Increasing P ext increases the mean pairwise correlation between the rates of the neurons in the network (feedforward network, Fig 1B).These correlations could be removed by recurrent connections.Initially, we set the excitatory recurrent synaptic amplitude as a exc = 0.01 nS to create a small perturbation from the feedforward network and varied the scaling α determining the amplitude of inhibitory synapses (a inh = αa exc ) from 15 to 25, which leads to the amplitude of inhibitory post-synaptic potentials being sever-fold (approximately 2× to 8×, depending on α and on the memory potential) larger than the excitatory post-synaptic potentials, as commonly chosen in network modelling [30,31,32,29].Correlations between neurons were decreased for α � 20 (Fig 1C ), which was also associated with stronger negative net current from the recurrent synapses (

Fano factor of single neurons vs. a population
In an inhibition-dominated network, the input needed from the external population in order to evoke a given average firing rate has to be higher than in the case of the feedforward network.The resulting increase in synaptic noise leads to higher Fano factor in the LIF model (Fig 2A, 2B and 2C; see also [33]).
If we assume that the downstream areas decode the stimulus intensity from the summed activity of the network, we need to look at the Fano factor of the summed activity, that is, ratio of variance of the sum to the mean of the sum across the trials of duration ΔT = 1 s.In the case of the total population activity, however, the pairwise correlations between the neurons have a significant effect on the Fano factor.By denoting the random variable representing the number of spikes of the i-th neuron observed during time window ΔT as N i , we get for the Fano factor of the population activity: where c is the mean pairwise covariance, v the mean variance of a neuron, μ is the mean number of spikes in ΔT, n tot is the number of neurons, and r is the Pearson correlation coefficient.The last approximation holds for neurons with identical variances and pairwise covariances [16].It provides an insight into how the pairwise correlations and Fano factor of individual neurons affect the Fano factor of the total activity.If the correlations or number of neurons are small (r � n tot � 1), the decorrelation by strengthening the recurrent synapses does not significantly decrease the population Fano factor.Instead, the population Fano factor may increase due to the increase of the Fano factor of individual neurons (Fig 2D, P ext = 0.01).If greater correlations are induced due to the shared input to the network, the correlations have a dominating effect on the population Fano factor, which can then be greatly decreased by strengthening the recurrent synapses and in turn decreasing the pairwise correlations (Fig 2E and 2F).

Inhibitory feedback is metabolically costly
2.4.1 Stronger recurrence strength increases the cost of the resting state.We calculated the cost of the activity by summing the cost of action potentials from the excitatory, inhibitory, and external subpopulations, and the cost of excitatory synaptic currents in the excitatory and inhibitory subpopulations.These excitatory currents may be evoked by action potentials from the external or excitatory subpopulations, or from the background input.We did not consider the cost of synaptic currents evoked in neurons not involved in our simulation.We assume that such synaptic currents would be part of the background activity of a different area.Therefore, if we included these costs and considered multiple cortical areas, we would have included the background activity cost multiple times.We also did not include the cost of synaptic currents in the external population.
The cost of the resting state is an important factor for information-metabolic efficiency [10].In our network, increasing the recurrence strength decreased the spontaneous activity of the neurons, due to inhibition dominating the recurrent currents.However, the simultaneous increase in the strength of the recurrent excitatory synapses increased the cost of the excitatory synaptic currents (Fig 3A, 3B and 3C), because the spontaneous action potentials from the excitatory subpopulation evoke stronger excitatory post-synaptic currents.

Inhibitory feedback decreases gain.
Because the net current from recurrent synapses is hyperpolarizing, with stronger recurrent synapses, a stronger excitatory current is necessary to bring the neuron to a given post-synaptic firing rate, and higher pre-synaptic firing rates are necessary.Therefore, the gain g of the network decreases, and with increasing a rec the cost of synaptic currents and the cost of external activity increase (Fig 3D and 3E).

Shared input decreases gain
The number of synapses from the external population for each neuron in the excitatory and inhibitory subpopulations follows a binomial distribution: with the mean number of synapses given by n ext � P ext and variance n ext � P ext (1 − P ext ).We scaled the firing rate of the individual neurons in the external population as l 0 exc ¼ l exc n ext �P ext .Therefore the mean output to a single neuron was always λ ext , independently of P ext and the variance of the input across neurons was l ext n ext 1À P ext P ext .Given the convexity of the single neuron tuning curve in the analyzed input range (S1 Fig) that out of two inputs with an identical mean λ ext , but different variances across neurons, the input with the higher variance will lead to a higher average firing rate.Assuming that the input across neurons follows a normal distribution with mean λ ext and variance σ 2 and that the single neuron tuning curve can be approximated by an exponential function in the form of c 1 exp (c 2 x), where x is the input intensity to the single neuron, we obtain the mean firing rate: which grows with the standard deviation of the input.Accordingly, we observed that networks with higher P ext needed higher λ ext in order to produce the same mean PSFR as networks with lower P ext (Fig 4A , 4B and 4C), which translates to lower gain with higher P ext (Fig 4D , 4E and 4F).Moreover, the mean Fano factor of individual

Optimal regimes for metabolically efficient information transmission
We illustrated that the recurrence strength 1) increases the metabolic cost of the neural activity and 2) decreases the population Fano factor by decreasing the correlations between the neurons.Similarly, the increased probability of a synapse from an external population (P ext ) decreases the cost of the neural activity but increases the noise correlations.The increased noise correlations then result in higher Fano factor (Eq 26).To find the balance between the cost of the network activity (Eq 4) and the mutual information between the input and the output (Eq 5), we calculated the information-metabolic efficiency, which maximizes the ratio of the mutual information to the cost of the network activity (Eq 10).
For low values of P ext (� 0.1), increasing the strength of the recurrent input did not lead to an increase in the information-metabolic efficiency.For higher values of P ext the informationmetabolic efficiency was maximized for a rec between 0.1 nS and 0.5 nS (Fig 5A and 5B), meaning that the strength of the recurrent excitatory synapses was 2× to 5× lower that the strength of the synapses from the external population.
Moreover, varying P ext had a significant effect on the information-metabolic efficiency across all values of a rec .Namely, low values of P ext resulted in lower values of information-metabolic efficiency across all values of a rec , showing that shared input from the external population is beneficial for metabolically efficient information transmission.Overall, the highest values of information-metabolic efficiency (E � 2bit/10 12 ATP) were reached for a rec between 0.05 nS and 0.5 nS and P ext between 0.2 and 1 (Fig 5B ).
We analyzed the effect of the resting cost (Fig 3A , 3B and 3C) by setting the resting cost in all cases equal to W 0 , the resting cost of the feedforward network.This did not have a significant effect on the information-metabolic efficiencies (S3 Fig).
Neural circuits might not necessarily maximize the ratio of information to cost.Instead, neurons and neural circuits could modulate their properties to maximize information transmission with the available energy resources [5].For example, neurons in the mouse visual cortex have been shown to decrease the conductance of their synaptic channels after food restriction [35].
Accordingly, we studied how the optimal strength of recurrent synapses changes with the available resources.We calculated the optimal value of a rec for different values of available resources (3,4,5,6,7,8,10,12,15,20,30, and 40 × 10 12 ATP).In Fig 5C , 5D, 5E, 5F, 5G and 5H, we plotted C(W; a rec ), the capacity-cost function (Eq 9) extended by one dimension with a rec .For each cost W, the optimal a rec is highlighted, and the corresponding contour of C(W) is shown (see Table 1 for the values of C(W)).With decreasing W, the optimal value of a rec typically decreases.This effect is more robust with high values of P ext , because the contours are more curved at the optimum.
We calculated the extended capacity-cost functions using input distributions obtained from the low-noise approximation.To verify that the low noise approximation applies in the case of the studied system, we compared these results to the information-metabolic efficiency obtained with the Jimbo-Kunisawa algorithm.The relative difference did not exceed 10% and did not have a significant impact on the information-metabolic efficiency heatmap structure (S4 Fig).

Limits of efficient information transmission by the population activity
So far we have assumed that the information about the stimulus is transmitted by the total activity of the network.Such analysis provides us with important insights, however, such simplistic decoding might not necessarily occur in the brain.To explore the limits of decoding the input intensity from the population activity, we assert that the brain can perform optimal unbiased decoding of the stimulus, i.e., for each stimulus λ ext , it holds for the estimation of the input l that where the second equation corresponds to an estimator which saturates the Crame ´r-Rao bound, and J pop (λ ext ) is the Fisher information about the stimulus from the population activity.
If we assume that l is distributed normally, we may then write the conditional probability distribution function as: obtaining a noisy identity channel with the noise given by the Crame ´r-Rao bound.
To reduce the effect of sampling bias, we estimated J pop from the first 500 principal components of the output and employed a bias correction (see section 4.4 for details).Increasing the strength of recurrent connections (a rec ) increased the information metabolic efficiency of the network (Fig 6).The increase was more pronounced with higher values of P ext , and overall was the highest for P ext = 0.8 and P ext = 1.In this sense, the results remain qualitatively very similar to the information-metabolic efficiency calculated from the summed activity (Fig 5).Interestingly, however, our results indicate that when using information from the entire population, not only the summed activity, the noise correlations introduced by the shared input are less detrimental, and P ext = 1 reaches the highest or close to highest values of the information-metabolic efficiency.

Discussion
Information in the brain is likely transmitted by neuronal populations instead of single neurons [19].One of the benefits is that by considering the signal from many neurons, it is possible to decrease the noise inherent to rate coding spiking neurons, and thus increase the information carried by the system.The information increase is however influenced by correlations between the neurons and their structure.In this work, we investigated a situation where a population of neurons tuned to the same stimulus transmits information about the stimulus intensity.In this case, positive noise correlations decrease the information carried by the population.
We parameterized the shared input with the probability of connection from the external population P ext .Higher P ext means that the firing rate of neurons in the external population can be lower to maintain the same mean input to the information-transmitting population.This way, the shared input, while increasing the noise correlations, decreases the metabolic cost of the activity.In the studied system, we could mitigate the noise correlations by strengthening the recurrent connections and thus increasing the inhibitory feedback.However, to excite a population with inhibitory feedback requires stronger input than to excite a population without inhibitory feedback, and therefore, strengthening the recurrent connection increased the cost of the activity.
In our work, we studied the balance between increasing the transmitted information by decreasing the noise correlations and the associated increase in the cost of the activity.We showed that in a linear system, if the Fano factor of the population activity and the ratio g w 0 FF (g is the gain of the system, or slope of the stimulus-response curve, w 0 is the slope of the stimulus-cost curve) remain constant, the cost-constrained capacity will remain constant as well.
We proceeded to calculate the stimulus-response relationship and the metabolic cost for a more biologically realistic neural system.In the studied system, the population Fano factor could not be considered constant.Instead, correlations between neurons increased with the mean output of the system, and the mean Fano factor of single neurons was also dependent on the mean output of the system, leading to complex dependence of the population Fano factor on the mean output of the system (Fig 2D , 2E and 2F).We found that despite increasing the noise correlations, the shared input helps with information-metabolically efficient information transmission.This was further accented if the noise correlations are decreased by the increase in the inhibitory feedback.Increasing the recurrence strength could lead to a 10% to 15% increase in the information-metabolic efficiency.The magnitude of the increase was dependent on the cost of the action potentials.If the cost of synaptic currents is negligible compared to the cost of the action potentials, there would be a higher benefit in increasing the inhibitory feedback since the increases in the cost of the synaptic current could also be neglected.
We illustrated the effect of inhibition-dominated recurrence and shared input on the metabolic cost of neural activity.An increased strength of recurrence increased the cost of excitatory synaptic currents due to the stronger excitatory synapses and stronger input from the external population, as well as the cost of the activity of the external population.A higher connection probability from the external population (higher shared input probability) led to a decrease in the external population activity cost, as the overall activity of the external population could be lower to result in the same mean input to the post-synaptic neurons.On the other hand, due to less variable input to single neurons with high values of P ext , a higher mean input was required across all neurons to evoke the same mean post-synaptic activity.
In our model of the cortical area, we considered two neural subpopulations: excitatory and inhibitory.Each subpopulation was homogeneous, but we set the threshold of the inhibitory neurons lower to mimic the behavior of fast-spiking inhibitory neurons.The difference between excitatory, regular spiking neurons and inhibitory, fast-spiking neurons is often described not only by differences in the threshold but also in differences in the adaptation properties [36,37,29].In our case, we did not consider adaptation for simplicity because estimating the information capacity of a neural system with adaptation is computationally considerably more difficult [10].
In our work, we assumed that the neural circuit maximizes the mutual information between the input and the output neurons while minimizing the cost of the neural activity.Such an approach does not provide any information about how the information is encoded.It only calculates the limit on the amount of information that can be transmitted.Yet, the principles of mutual information maximization have proven very useful in explaining the properties of neural systems.For example, the tuning curves of blowfly's contrast-sensitive neurons are adapted to the distribution of contrasts encountered in the natural environment [38]; the power spectrum of distribution of odor in pheromone plumes follows the power spectrum predicted for an optimal input to olfactory receptor neurons [39]; distributions of post-synaptic firing rates of single neurons during in-vivo recordings follow distributions predicted from cost-constrained mutual information maximization [40,41,42].
By assuming a particular coding scheme, it is possible to place further constraints on the complexity of information encoding, with the assumption that complex codes are not an efficient way to transmit information [43,44].We did not attempt this in our study.However, it would be interesting to study whether inhibitory feedback decreases or increases the encoding complexity.
We have shown that a cortical area can adapt to the amount of available energy resources.When resources are scarce, information transmission can be adapted by weakening the synaptic weights, thus expending fewer resources to reduce the noise correlations.Such a mechanism is implemented in the mouse visual cortex [35].Padamsey et al. [35] showed that in food-restricted mice, the orientation tuning curves of individual orientation-sensitive neurons in the visual cortex become broader due to weakened synaptic conductances.In our work, we studied the properties of a neuronal population instead of single neurons.In particular, we considered a population encoding the stimulus intensity instead of the stimulus identity, such as orientation.An extension this model to a situation in which stimulus identity is encoded and shared input is introduced due to the overlap of receptive fields would be interesting.
Neurons recorded in-vivo typically exhibit a Fano factor close to 1.0 and constant over a broad range of post-synaptic firing rates [45,46,19].In the optimal regimes with stronger recurrent synapses, the Fano factor decreased only very slowly over the studied range of postsynaptic firing rates (up to 30 Hz in a single neuron).With weaker synaptic strengths, the Fano factor of a single neuron decreases rapidly with an increasing post-synaptic firing rate.Our model predicts that fewer available resources would lead to weaker recurrent synapses.This hypothesis is straightforward to test by calculating the Fano factors during stimulus presentation (both population and single neuron) in food-restricted animals and comparing them to controls.We expect that the population Fano factor will increase (alternatively, the noise correlations will increase) with food scarcity, and single neuron Fano factors will decrease.

Network model
We modeled a network consisting of three subpopulations: external (ext), excitatory (exc), and inhibitory (inh).The external subpopulation consisted of Poisson neurons, defined by their firing intensity l 0 ext (same for all the neurons in the subpopulation).Neurons in the excitatory and inhibitory subpopulations were modeled as leaky integrate-and-fire (LIF) neurons: I rec is the synaptic current arising from the recurrent connections (exc.to exc., exc. to inh., inh. to exc., inh. to inh.).I ext is the excitatory current from external neurons.I bcg is the current from synapses from neighboring cortex areas.T j ext , T j exc , T j inh represent the spike times of the jth external, excitatory, and inhibitory neuron respectively.The matrices W ext , W exc , W inh contain the synaptic connection strengths, W ij X ¼ a X (X 2 {ext, exc, inh}) if the j-th neuron connects to the i-th neuron and 0 otherwise.The background (bcg) input from neighboring cortical areas is modeled as the Ornstein-Uhlenbeck process with means μ bcg,exc and μ bcg,inh and standard deviations of the limiting distributions σ bcg,exc and σ bcg,inh [47,48].We set the values of the background activity to match the moments of an exponential Poisson shot noise with rates λ bcg,exc = 0.5 kHz and λ bcg,inh = 0.125 kHz [49]: where X represents the excitatory or inhibitory background activity, leading to the ratio of inhibitory to excitatory conductance of gl bcg;inh l bcg;exc ¼ 5, as observed in-vivo [48] and a spontaneous firing rate of about 0.5 Hz to 1 Hz.
When the membrane potential V crosses the firing threshold (θ exc , θ inh ) a spike is fired and the membrane potential is reset to E L .
The network consisted of n ext = 1000 neurons in the external population, n exc = 800 neurons in the excitatory population, and n inh = 200 neurons in the inhibitory population.The connections were set randomly with connection probability for the recurrent connections (exc.to exc., exc. to inh., inh. to inh., inh. to exc.) set to P rec = 0.2 and the connection probability from the external population (ext.to exc. and ext. to inh., P ext ) was varied from to 0.01 to 1 (Fig 1A).We created the connection matrices W X by generating a matrix of random uniformly distributed numbers R X the interval [0, 1) and set W ij X ¼ a X if R ij ext < P ext or R ij X < P X for X 2 {exc, inh}.The random matrix R ext was the same for all values of P ext .In simulations where we controlled for the effects caused by a random number of connections from the external population, we fixed the number of connections by setting only the k = n ext P ext elements in each row of W ext non-zero, in the location of the k largest elements of the i-th row of R ext .
The simulations were carried out using the Brian 2 package [50] in Python with a 0.1 ms time step.Used parameters are given in Table 2.

Obtaining the input-output relationship of the network
We considered the total number of action potentials n from the excitatory and inhibitory subpopulations in time window ΔT = 1 s as the output of the network.We modeled the stimulus as the input from the thalamic neurons, parametrized by the mean input rate to a single neuron: where l 0 ext is the firing rate of a single neuron in the external population, n ext l 0 ext is the input firing rate at P ext = 1, and 1 P ext is a scaling factor to keep the mean input same regardless of P ext .For each set of parameters (a rec and P ext pair) we determined the input l max ext ða rec ; P ext Þ for which the output reached 30 kHz.In order to obtain the input-output relationship, we discretized the input space into 30 equidistant stimulus intensities: where i = 0, . .., 30.With a fixed network connectivity, we simulated the network 10800 times for each l i ext ða rec ; P ext Þ.We discretized the input space to 1000 equidistant stimulus intensities and estimated the mean output μ(λ ext ) and variance σ 2 (λ ext ) for each intensity by linear interpolation from the simulated data.We then estimated the input-output relationship, defined by the conditional probability distribution f(n|λ ext ) as a discretized normal distribution for each λ ext , with corresponding mean and variance:

Metabolic cost of neural activity
In our calculations, we focus on the energy in the form of ATP molecules required to pump out Na + ions.We take into account the Na + influx due to excitatory post-synaptic currents, Na + influx during action potentials, and Na + influx to maintain the resting potential.To this end, we follow the calculations in [2] and [3], which we modify for our neuronal model.We assume the standard membrane capacitance per area as c m = 1 μF/cm 2 and the cell diameter as D = 69 μm, giving the total capacitance C m = πD 2 c m = 150 pF.Therefore, to depolarize a neuron by ΔV = 100 mV the minimum charge influx is ΔVC m = 1.5 × 10 −11 C and the minimum number of Na + ions DVC m e ¼ : 9:375 � 10 7 , where e ¼ : 1.6 × 10 −19 is the elementary charge.The minimal number of Na + ions is then quadrupled to get a more realistic estimate of the Na + influx due to the simultaneous opening of the K + channels [2].The Na + influx must be then pumped out by the Na + /K + -ATPase, which requires one ATP molecule per 3 Na + ions.The cost of a single action potential can be then estimated as 4 3 � 9:375 � 10 7 ATP ¼ 1:25 � 10 8 ATP.However, about 75% of the metabolic costs associated with an action potential are expected to come from the propagation of the action potential through the neuron's axons [51,2].Therefore, we estimate the total cost as 5.0 × 10 8 ATP.Next, we assume that the excitatory synaptic current is mediated by the opening of Na + and K + channels with reversal potentials E Na = 90 mV and E K = −105 mV.For the excitatory synaptic current, the following must hold Therefore: The sodium entering with the sodium current I Na must be pumped out by the Na + /K + -ATPase and therefore we calculate the cost of the synaptic current as 1 3e I Na DT ATP, where ΔT is the time interval over which we are measuring the cost.
Each input to the network (parametrized by λ ext ) is then associated with a cost, which we express as where μ exc = μ exc (λ ext ), μ inh = μ inh (λ ext ) are the mean firing rates of a single excitatory and inhibitory neuron (given the input λ ext ), hI exc Na i ¼ hI exc Na iðl ext Þ and hI inh Na i ¼ hI inh Na iðl ext Þ are the average excitatory synaptic currents in a single excitatory and inhibitory neuron.

Fisher information with multidimensional output
When we consider that the output of the network is either the full vector of firing rates, or its low-dimensional projection, we can calculate the Fisher information as Tr S À 1 @S À 1 @l ext S À 1 @S À 1 @l ext � � ; ð50Þ where f(λ ext ) is the mean of the multidimensional response vector, S(λ ext ) (dependence of S was omitted for legibility) is the covariance matrix of the response components at input λ ext , and Tr stands for the Trace operator.The first term in the equation is analogous to the Fisher information in one-dimensional case (Eq 14), while the second term indicates how much information we gain about the stimulus from changes in the covariance matrix.In our case, the second term was always very small compared to the first term.We performed dimensionality reduction of the output across all stimuli by principal component analysis and used the first 500 principal components.We used 500, because the increase in information-metabolic efficiency for higher number of components is small, and the sampling bias is still relatively small (S5 Fig) .To deal with the remaining sampling bias we calculated the information-metabolic efficiency with the Jimbo-Kunisawa for different numbers of trials and performed the quadratic extrapolation method to estimate the unbiased information-metabolic efficiency [52,53].Overall, the results remain qualitatively very similar to the information-metabolic efficiency calculated from the summed activity.However, we found the increase in information-metabolic efficiency from using high-dimensional output is the largest for higher values a rec and P ext .
4.4.1 Correcting the sampling bias.In the case of a high-dimensional output, insufficient number of trials may lead to perceived correlations in the data which are in fact not there, subsequently increasing the calculated mutual information [54,52,55,56,53].To decrease the sampling bias, we first performed principal component analysis to decrease dimensionality of the output and employed a quadratic extrapolation method to estimate the unbiased value of information-metabolic efficiency.We used the Jimbo-Kunisawa algorithm to calculate information-metabolic efficiency with 10800, 5400, and 2700 trials, obtaining the estimates of E (Eq 10): E 10800 , E 5400 , and E 2700 .We then assumed that the estimates follow the following dependency on the number of trials k [52]: By solving the linear system we obtained the estimate of the unbiased information-metabolic efficiency E 0 (S5 Fig) .We found that with 500 principal components the bias is still relatively low, and further increasing the number of components leads only to minor increase in the information-metabolic efficiency.Therefore, we used the first 500 components to obtain the results in the Fig 6.
Fig 1D).For the network considered further in our work we set α = 20.Simultaneously increasing the strength of the recurrent synapses with fixed α led to a further decrease of the correlations among the neurons (Fig 1E) while further decreasing the net current from the recurrent synapses (Fig 1F).

Fig 1 .
Fig 1. Inhibitory feedback decreases noise correlations.A: Schematic illustration of the simulated neural network.Poisson neurons in the external population make random connections to neurons in the excitatory and inhibitory subpopulations.The connection probability P ext 2 [0.01, 1] is varied to achieve different levels of shared external input to the neurons.The neurons in the inhibitory (inh.) and excitatory (exc.)subpopulations make recurrent connections (exc.to exc., exc. to inh., inh. to inh., inh. to exc.) with probability P rec = 0.2.The strength of those connections is parametrized by a rec .B: Mean pairwise correlations between any two neurons in the exc.and inh.subpopulations plotted against the mean output of the network for different values of P ext in a feedforward network (a rec = 0 nS).Pairwise correlations are calculated from the number of spikes each neuron fires in a time window ΔT = 1 s across many trials of the simulation.The plot is vertically separated into two parts to also illustrate the smaller differences at lower values of P ext .C: Mean pairwise correlations as in B, for different values of α (ratio of inhibitory-to-excitatory synaptic strength), a rec = 0.01 nS.The black line represents the pairwise correlations in a feedforward network without any recurrent connections (a rec = 0).D: Total current from recurrent synapses for different values of α, as in C. E-F: Same as in C-D, but with fixed α = 20 and different values of a rec .https://doi.org/10.1371/journal.pcbi.1011896.g001

Fig 2 .
Fig 2. Fano factor of single neurons and of populations.A-C: Mean Fano factor of individual neurons for different values of P ext : 0.01 (A), 0.2 (B), 1 (C).The strength of the recurrent synapses (a rec ) is color-coded.The mean Fano factor increases with the strength of the recurrent synapses.D-F: Same as in A-C but for the Fano factor of the population activity.The points represent the population Fano factor obtained from the simulation, and the lines are a weighted 7th-degree polynomial, used only as a visual aid.For P ext = 0.01, the increase in Fano factor of individual neurons (A) can have a stronger effect on the population Fano factor than decreasing the pairwise correlations, resulting in an increase of the population Fano factor with high values of a rec (D).For higher values of P ext , the pairwise correlations greatly increase the population Fano factor, which then decreases with increasing a rec .https://doi.org/10.1371/journal.pcbi.1011896.g002

Fig 3 .
Fig 3. Metabolic cost of the network activity.A-C: Cost at resting state (λ ext = 0).A: Cost of the excitatory synaptic currents from the background input (Eq 35) and excitatory action potentials evoked by the background input.B: Cost of the action potentials (both excitatory and inhibitory) evoked by the background input.C: Total resting cost obtained by summing A and B. D: The total cost of the network activity is plotted against the output of the network (the total post-synaptic firing rate).Filled areas represent individual contributions of each cost component: cost of action potentials from the external population, cost of the excitatory synaptic currents, and cost of the post-synaptic (evoked) action potentials.As P ext increases, the contribution of external action potentials to the overall cost decreases.With increasing a rec , the contribution of excitatory synaptic currents increases.E: The cost of increasing the mean input by one action potential (w AP , Eq 19) is significantly lower for higher P ext .However, although the difference between P ext = 0.01 and P ext = 0.2 is approximately 10-fold, the difference between P ext = 0.2 and P ext = 1 is only approximately 2-fold, as the cost of the external population starts to contribute less to the overall cost.https://doi.org/10.1371/journal.pcbi.1011896.g003

Fig 4 .
Fig 4. Shared input decreases the gain and increases the individual Fano factor.A-C: The input intensity λ ext needed to evoke a given firing rate (x-axis) with different connection probabilities P ext relative to the input intensity for P ext = 0.01.A: a rec = 0 nS, B: a rec = 0.2 nS, C: a rec = 1 nS.For higher P ext , higher values of λ ext are needed to achieve the same post-synaptic firing rates as with lower values of P ext .This effect becomes more pronounced in stronger recurrent synapses (E-F).D-F: Gain of the network (Eq 20).A higher P ext leads to a lower gain of the population activity.G-I: Higher values of P ext also increase the Fano factor of individual neurons.https://doi.org/10.1371/journal.pcbi.1011896.g004